AMENDMENTS TO THE CLAIMS 

1 . (Currently Amended) A method of real-time rendering in a gaming 
environment to create an interactive experience, comprising: 

capturing a plurality of real-time video streams of a local participant from a 
plurality of camera viewpoints; 

generating real-time renderings of said local participant based on said 
plurality of real-time video streams by applying a new view synthesis technique, 
said real-time renderings taken from a perspective of a remote participant 
located remotely in said gaming environmen t, wherein said new view synthesis 
technique comprises an image-based visual hull technique that comprises 
approximating a visual hull of said local participant by projecting contours 
associated with said local participant into three-dimensional space and 
computing an intersection of resulting frusta ; and 

sending said real-time renderings to said remote participant for viewing 
within said gaming environment. 

2. (Original) The method of Claim 1 , further comprising: 
incorporating said real-time renderings comprising real-time photo-realistic 

features of said local participant into a graphically generated avatar, said 
graphically generated avatar representing said local participant in said gaming 
environment; and 

sending said graphically generated avatar incorporating said real-time 
renderings to said local participant for viewing within said gaming environment. 

3. (Canceled). 
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4. (Original) The method as described in Claim 1 , wherein said 
generating real-time renderings of said local participant further comprises: 

generating a three-dimensional geometric model of said local participant 
from said plurality of real-time video streams as said real-time renderings. 

5. (Original) The method as described in Claim 1 , wherein said 
generating real-time renderings of said local participant further comprises: 

rendering said real-time renderings from a second location of said remote 
participant with respect to a first location of said local participant within a 
coordinate space of said gaming environment. 

6. (Original) The method as described in Claim 5, further comprising: 
enabling a change in at least one of said first and second locations by 

allowing said local participant and said remote participant to navigate through 
said coordinate space; and 

modifying said real-time renderings to reflect said change in at least one 
of said first and second locations. 

7. (Currently Amended) The method as described in Claim 1 , further 
comprising: 

blending said real-time renderings within a synthetic rendering of said 
gaming environment; and 

displaying said real-time renderings that are is blended within said gaming 
environment at a display viewable by said second participant to enable real-time 
video communication for interactive gaming between said local participant and 
said remote participant within said gaming environment. 
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8. (Original) The method as described in Claim 1 , wherein said gaming 
environment comprises a three dimensional gaming environment. 

9. (Original) The method as described in Claim 1 , further comprising: 
personifying said graphically generated avatar to said local participant by 

illustrating real-time emotion of said local participant through said video image 
stream comprising representative said real-time photo-realistic features of said 
local participant. 

10. (Currently Amended) The method as described in Claim 1, further 
comprising: 

generating other another real-time renderings of said local participant 
based on said plurality of real-time video streams by applying said new view 
synthesis technique, said other anoth e r real-time renderings taken from a 
perspective of another participant located remotely in said gaming environment; 
and 

sending said other anoth e r real-time renderings to said another participant 
for viewing within said gaming environment. 

1 1 . (Currently Amended) A method of real-time rendering in a gaming 
environment to create an interactive experience, comprising: 

capturing a plurality of real-time video streams of a local participant from a 
plurality of camera viewpoints; 

generating a video image stream in real-time of said local participant 
based on said plurality of real-time video streams by applying a new view 
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synthesis technique, said video image stream rendered from a perspective of a 
remote participant located remotely in said gaming environment , wherein said 
new view synthesis technique comprises an image-based visual hull technique 
that comprises approximating a visual hull of said local participant by projecting 
contours associated with said local participant into three-dimensional space and 
computing an intersection of resulting frusta ; and 

incorporating said video image stream comprising real-time photo-realistic 
features of said local participant into a graphically generated avatar, said 
graphically generated avatar representing said local participant in said gaming 
environment. 

12. (Original) The method as described in Claim 1 1 , wherein said 
generating said video image stream further comprises: 

rendering said video image stream from a second location of said remote 
participant with respect to a first location of said local participant within a 
coordinate space of said gaming environment 

13. (Original) The method as described in Claim 12, further comprising: 
enabling a change in at least one of said first and second locations by 

allowing said local participant and said remote participant to navigate through 
said coordinate space; and 

modifying said video image stream to reflect said change in at least one of 
said first and second locations. 
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14. (Original) The method as described in Claim 11, further comprising: 
blending said graphically generated avatar incorporating said video image 

stream within a synthetic rendering of said gaming environment; and 

displaying said graphically generated avatar incorporating said video 
image stream within said gaming environment at a display viewable by said 
remote participant to enable real-time video communication for interactive 
gaming between said local participant and said remote participant within said 
gaming environment. 

15. (Original) The method as described in Claim 11, further comprising: 
personifying said graphically generated avatar to said local participant by 

illustrating real-time emotion of said local participant through said video image 
stream comprising representative said real-time photo-realistic features of said 
local participant. 

16. (Canceled). 

17. (Original) The method as described in Claim 11, wherein said 
generating said video image stream further comprises: 

generating a three dimensional geometric model of said local participant 
based on said plurality of real time video streams by applying said new view 
synthesis technique; and 

generating said video image stream of said local participant from said 
three dimensional geometric model. 



200310011-1 
Examiner: KIM, A. 



6 



Serial No.: 10/688,148 
Group Art Unit: 3712 



18. (Original) The method as described in Claim 1 1 , further comprising: 
generating a real-time audio output from audio of said local participant to 

enable real-time audio communication in said gaming environment between said 
local participant and said remote participant. 

19. (Original) The method as described in Claim 1 1 , further comprising: 
varying a total of said plurality of camera viewpoints when capturing said 

plurality of real-time video streams of said local participant to vary image quality 
in said plurality of video image streams. 

20. (Original) The method as described in Claim 11, further comprising: 
performing non-photorealistic processing of said plurality of video image 

streams resulting in a non-photorealistic cartoon rendering of said plurality of 
video image streams. 

21. (Currently Amended) A gaming station comprising: 

a plurality of video cameras for recording real-time video streams of a 
local participant from multiple sample viewpoints surrounding said local 
participant; 

a new view synthesis module for generating a plurality of output real-time 
renderings of said local participant [[,]] by applying a new view synthesis 
technique to said real-time video streams, wherein said output real-time 
renderings are taken from a plurality of locations within a coordinate space of a 
three dimensional gaming environment , wherein said new view synthesis 
technique comprises an image-based visual hull technique that comprises 
approximating a visual hull of said local participant by projecting contours 
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associated with said local participant into three-dimensional space and 
computing an intersection of resulting frusta . 

22. (Original) The gaming station of Claim 21 , further comprising: 
an interface module for incorporating a plurality of input video image 

streams comprising real-time photo-realistic features of a plurality of observing 
participants into a plurality of graphically generated avatars representing said 
plurality of observing participants, and blending said plurality of graphically 
generated avatars within a synthetic rendering of said three dimensional gaming 
environment. 

23. (Original) The gaming station as described in Claim 22, wherein said 
interface module receives a plurality of input audio streams in real-time 
associated with said plurality of observing participants for mixing into a single 
audio stream that is broadcast through a local speaker. 

24. (Original) The gaming station as described in Claim 22, further 
comprising: 

a display for viewing said plurality of graphically generated avatars 
representing said plurality of observing participants to enable real-time view 
communication between said local participant and said plurality of observing 
participants within said three dimensional gaming environment. 

25. (Original) The gaming station as described in Claim 21 , further 
comprising a contour extraction module for extracting said local participant from 
a physical background. 
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26. (Original) The gaming station as described in Claim 21, further 
comprising a means for capturing an audio stream in real-time of said local 
participant. 

27. (Currently Amended) A computer system comprising: 
a processor; and 

a computer readable memory coupled to said processor and containing 
program instructions that, when executed, implement a method of real-time 
rendering in a gaming environment to create an interactive experience, 
comprising: 

capturing a plurality of real-time video streams of a local participant from a 
plurality of camera viewpoints; 

generating real-time renderings of said local participant based on said 
plurality of real-time video streams by applying a new view synthesis technique, 
said real-time renderings taken from a perspective of a remote participant 
located remotely in said gaming environmen t, wherein said new view synthesis 
technique comprises an image-based visual hull technique that comprises 
approximating a visual hull of said local participant by projecting contours 
associated with said local participant into three-dimensional space and 
computing an intersection of resulting frusta ; and 

sending said real-time renderings to said remote participant for viewing 
within said gaming environment. 
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28. (Original) The computer system of Claim 27, wherein said method 
further comprises: 

incorporating said real-time renderings comprising real-time photo-realistic 
features of said local participant into a graphically generated avatar, said 
graphically generated avatar representing said local participant in said gaming 
environment; and 

sending said graphically generated avatar incorporating said real-time 
renderings to said local participant for viewing within said gaming environment. 

29. (Canceled). 

30. (Original) The computer system as described in Claim 27, wherein 
said generating real-time renderings of said local participant in said method 
further comprises: 

generating a three-dimensional geometric model of said local participant 
from said plurality of real-time video streams as said real-time renderings. 

31 . (Original) The computer system as described in Claim 27, wherein 
said generating real-time renderings of said local participant in said method 
further comprises: 

rendering said real-time renderings from a second location of said remote 
participant with respect to a first location of said local participant within a 
coordinate space of said gaming environment. 
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32. (Original) The computer system as described in Claim 31 , wherein 
said method further comprises: 

enabling a change in at least one of said first and second locations by 
allowing said local participant and said remote participant to navigate through 
said coordinate space; and 

modifying said real-time renderings to reflect said change in at least one 
of said first and second locations. 

33. (Currently Amended) The computer system as described in Claim 27, 
wherein said method further comprises: 

blending said real-time renderings within a synthetic rendering of said 
gaming environment; and 

displaying said real-time renderings that are is blended within said gaming 
environment at a display viewable by said second participant to enable real-time 
video communication for interactive gaming between said local participant and 
said remote participant within said gaming environment. 

34. (Original) The computer system as described in Claim 27, wherein 
said gaming environment comprises a three dimensional gaming environment. 

35. (Original) The computer system as described in Claim 27, wherein 
said method further comprises: 

personifying said graphically generated avatar to said local participant by 
illustrating real-time emotion of said local participant through said video image 
stream comprising representative said real-time photo-realistic features of said 
local participant. 
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36. (Currently Amended) The computer system as described in Claim 27, 
wherein said method further comprises: 

generating other another real-time renderings of said local participant 
based on said plurality of real-time video streams by applying said new view 
synthesis technique, said other anoth e r real-time renderings taken from a 
perspective of another participant located remotely in said gaming environment; 
and 

sending said other anoth e r real-time renderings to said another participant 
for viewing within said gaming environment. 

37. (New) The method of Claim 1 , further comprising: 
obtaining parameters using a single uncalibrated target to calibrate 

cameras used for said capturing. 

38. (New) The method of Claim 2, further comprising: 

directing movement of said graphically generated avatar according to said 
local participant's eye gaze. 
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